Transforming supervised classifiers for feature extraction

Supervised feature extraction is used in data classification and, unlike unsupervised feature extraction, uses class labels to evaluate the quality of the extracted features. It can be computationally ineflcient to perform exhaustive searches to find optimal subsets of features. This article proposes an algorithm of supervised linear feature extraction based on the use of multivariate decision trees. The main motivation in proposing this new approach to feature extraction is to decrease computation time required to induce new classifiers required to evaluate every new subset of features. The new feature extraction algorithm proposed here uses an approach similar to the wrapper model method used in feature selection. In order to evaluate the performance of proposed algorithm, several tests with real world data have been performed. The fundamental importance of this new feature extraction method is found in its ability to sign ficantly reduce computational time required to extract features from large databases.